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ABSTRACT 



Asserting that in order to determine the effectiveness of 
those in leadership positions accurately and fairly, an understanding of how 
individuals and groups of individuals construct their notions of effective 
leadership within complex organizations must be developed. This study sought 
to: (1) define and measure effective leadership empirically (in this case, 

the effectiveness of deans of colleges in one university) and (2) examine 
individual and group characteristics that may affect the perceptions of 
effective leadership within the academic organization. Faculty and staff were 
surveyed on several dimensions of their dean's leadership effectiveness: 
vision and goal setting, management of the unit, interpersonal relationships, 
communication skills, research/professional/community endeavors, quality of 
the unit's education, and support for institutional diversity. Demographic 
data on respondents and deans and structural variables of the academic units 
were also included in the analysis. Findings indicated that unit size and 
external dollars generated are positively related to perceptions of 
leadership effectiveness; that women are viewed as better leaders than men; 
and that the respondent's place in the organization affects his or her 
perception of the dean's leadership effectiveness. (Contains 67 references.) 
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MAPPING THE DOMAINS OF EFFECTIVE LEADERSHIP: 
THE CASE OF DEANS AND DIRECTORS 



Despite countless efforts to define and understand leadership, it remains an oft-debated 
and even controversial construct. Explaining effective leadership is even more elusive and 
continues to intrigue and challenge the best of scholars, administrative teams, consultants and, 
most of all, leaders themselves. Leaders are deemed effective— or ineffective— by informal 
assessments of their leadership style, the performance of their duties and responsibilities, and 
even, at times, their individual traits or qualities. Such judgments flourish in organizations, and 
higher education is no exception. Faculty and staff members frequently comment on the 
effectiveness of their campus leaders based upon a gambit of personal perceptions. 

It may be argued, however, that the effectiveness of leaders in higher education is, for the 
most part, a matter of perception (Fincher, 1996). There are few shared norms about appropriate 
outcome measures for leaders in higher education, as opposed, for example, to business in which 
leaders’ performance may be assessed based on profit and growth. In fact, in colleges and 
universities there is no commonly accepted definition of effective leadership, and even less 
agreement about which aspects of a definition may be most important to effectiveness 
(Bensimon, Neumann, & Bimbaum, 1989; Bimbaum, 1992; Dill, 1984; Fincher, 1996; Neuman 
& Bensimon, 1990; Whetten & Cameron, 1985). 

Individual perceptions of effectiveness are based on what leaders say and do; that is, 
perceptions are grounded in the individual’s experience with the leader’s behavior, either directly 
or indirectly. From these experiences, individuals determine whether they believe leaders are 
effective or ineffective (Bimbaum, 1989; Fincher, 1996; Whetten & Cameron, 1985). 

Perceptions then are crucial to the viability of the leader’s position within the institution. 
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Perceptions may even be collected from a defined group of individuals in order to “evaluate” the 
leader’s performance. Such measurement of perceptions may constitute a “high stakes” 
evaluation for the individual leader (Heck, Johnsrud, & Rosser, in press), because they may lead 
to decisions about promotion, salary augmentation, contract renewal, or dismissal. 

In order to determine the effectiveness of those in leadership positions accurately and 
fairly, we must understand how individuals, as well as groups of individuals, construct their 
notions of effective leadership within complex organizations. Therefore, the purpose of this study 
is first to define and measure effective leadership empirically (in this case, the effectiveness of 
deans of colleges in one university) and second, to examine individual and group characteristics 
that may affect the perceptions of effective leadership within the academic organization. 

Effective Leadership 

One of the problems with the concept of leadership is the ambiguity of its definition and 
measurement (Pfeffer, 1977, 1978). In and of itself, leadership is among the most studied and 
least understood subjects (Bennis & Nanus, 1985; Bums, 1978). Leadership in organizational 
contexts is complex and multidimensional (Yukl, 1989, 1993). That complexity has attracted 
scholars from a variety of disciplines who bring contrasting perspectives to the subject. For 
example, scholars of psychology tend to focus on the individual and view leadership as an 
outcome of managerial effectiveness, success in influencing people, and developing commitment 
to task objectives (Yukl, 1989). Similarly, Chemers (1993) defines “leadership” as a process of 
social influence, and “effective leadership” as the successful application of the influence to 
mission accomplishment. He believes that effective leaders are able to obtain the cooperation of 
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other people and to harness the resources provided by that cooperation to the attainment of goals. 
Moreover, Boyatzis (1982) defines the effective performance of a leader as the attainment of 
specific results (i.e., outcomes) required by the job through specific actions while maintaining or 
being consistent with policies, procedures, and conditions of the organizational environment. 

A more sociological perspective conceives of leadership effectiveness as the degree of 
success with which a group performs the primary assigned task (Fiedler & Garcia, 1987). Fiedler 
and Garcia contend that, if reliable and objective measures of the group’s performance on tasks 
are not available, other types of outcome measures such as morale and satisfaction within the 
work unit may be important organizational factors in measuring effective leadership. Similarly, 
Bryman (1986) refers to effective leadership as a two-way influence process and notes that the 
leader must be responsive to the group’s position, as well as to the organization’s goals to be 
viewed as legitimate. Hence, leadership implies a relational dynamic that depends on the 
properties of the influencing leader, the situation or task, and the group being influenced. These 
conceptions define leadership as a mutually dependent process in which the leader is able to 
gamer group support to achieve specified organizational goals. 

Conventional leadership approaches place considerable weight on the behavioral displays 
of the leader (Meindl, 1993). In a social psychological approach, Meindl distinguishes between 
process and outcome measures of leadership effectiveness. He argues that behavioral displays of 
the leader are essentially “process” measures of effective leadership: they are not outcomes of 
value in and of themselves, but valuable in terms of their intended or likely impact on group 
productivity and the satisfaction of members. In contrast, group performance and behavioral 
displays of group members are significant “outcome” measures of leadership. Meindl (1990) 
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argues that performance cues, such as the behavioral displays of the leader, have been known to 
alter group members’ reports of leadership. He suggests that individuals have formed implicit 
theories and cognitive prototypes of what effective and ineffective leadership is defined in 
behavioral terms (p. 100). Meindl contends that this view is symptomatic of the emergence of 
leadership as a way of thinking on the part of group members. 

Leadership as a function of group perception is also reflected in Hollander’s (1964, 1978, 
1993) work, in that he believes that leadership effectiveness involves a group process with the 
leader as the directive element. Hollander argues that the process involves two-way influence 
aimed primarily at attaining mutual goals, such as those of a group, organization, or society. 
Followers are responsive to what leaders say and do, and leaders are responsive to followers. 
Similarly, Clark and Clark (1990) view effective leadership as a central concept used in studies 
of group dynamics, with the selection of leaders within a group being part of the method, as well 
as part of the outcome, of the research. Accordingly, leader effectiveness depends upon an equity 
in social exchange with the leader gaining status and exercising influence while helping the 
group to achieve desired mutual outcomes, as well as the individual leader achieving social 
rewards, such as recognition (Hollander, 1964, 1978). Hollander contends that this social 
exchange, or transactional approach to effective leadership, involves a trading of benefits. 

The conceptual foundations of transactional social exchange appear highly adaptable to 
certain features of academic organizations, such as governance as a collective process that 
involves all important constituencies, with particular emphasis given to the participation of the 
faculty (Bensimon et al., 1989). Within the academy such principles as shared governance, 
consultation, and leaders who are “first among equals” undergird the normative values 
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(Bensimon et al., 1989; Martin & Samels, 1997). Transactional social exchange attends to these 
multiple foundations of leadership and thus appears useful in understanding effective leadership 
of academic deans in higher educational institutions. 

Academic Deans 

College deans have the ability to exert power, control information, allocate resources, and 
assess the performance and promotions of their faculty and staff. They serve as academic 
facilitators between presidential initiatives, faculty governance, and student needs (Astin & 
Scherrei, 1980). As such, they are often referred to as the academic midlevel administrators in 
higher education (Morris, 1981; Roaden, 1970). By virtue of their midlevel placement within the 
organizational structure, they are in the center of controversy, conflict, and debate; they play the 
role of coalition builder, negotiator, and facilitator. Dill (1980) contends that the mid- 
management position of deans in most institutions is an amorphous, variegated and, perhaps 
ultimately, indescribable role. Nonetheless, efforts to describe deans include: “doves of peace” 
intervening among warring factions, “dragons” holding internal and external threats at bay, and 
“diplomats” guiding and encouraging people who live and work in the college (Tucker & Bryan, 
1991, p. ix). 

Even though individual deans have achieved remarkable power and status, there are many 
signs pointing toward an ebbing of powers (Dill, 1980). The once held vision of the dean as a 
quiet scholarly leader has been replaced by an executive image of the dean as politically astute 
and economically savvy (Gmelch et al., 1999). The deans’ delicate balancing act of their varying 
roles and responsibilities are often viewed differently by faculty, provosts, students, and deans 
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themselves (Gmelch et al., 1999). These differing perceptions of the position place multiple, 
conflicting, and more recently, consequential pressures upon deans (Gmelch et al., 1999; Kapel 
& Dejnozka, 1979). Dill contends that the deanship needs help and that new demands from 
various constituencies for miracles of performance are being added to old expectations, which 
were presumptuous enough (p. 262)! 

Evaluating the Effectiveness of Deans . Research on deans or “deaning” has been 
primarily descriptive in nature and refers to specific tasks and challenges (Gould, 1964; Morris, 
1981; Tucker & Bryan, 1991) and or the roles and functions of deans (Dibden, 1968; Dill, 1980; 
Griffiths & McCarty, 1980; Lasley & Haberman, 1987; Morris, 1981; Seldin, 1988; Tucker & 
Bryan, 1991). Rather than providing empirical studies that address the effectiveness of their 
leadership, most authors focus on specific topics such as their transition from research and 
teaching to academic management (Arter, 1981); their dilemmas in leadership (Cleveland, 1960); 
their position of conflict (Feltner & Goodsell, 1972); their management skills and mobility 
(Sagaria, 1988; Sagaria & Krotseng, 1986); their career paths (Moore et al., 1983); and their role 
in governance and decision making (Baldridge, 1971). There is, however, little empirical 
research that measures their effectiveness as leaders, least of all, as perceived by their faculty and 
administrative staff. 

In one recent study of dean’s perceptions of their own effectiveness, Wolverton et al. 
(1999) found that role ambiguity (e.g., knowing job responsibilities, having clear goals for the 
job, knowing amount of authority, knowing senior administrators’ expectations) had a small 
negative effect on deans’ perceptions of their job satisfaction, effectiveness, and commitment to 
the institution. The suggestion is that the less well articulated the role is, the less effective deans 
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perceive themselves to be. The implication for assessing deans’ effectiveness is evident: senior 
administrators need to articulate clearly the job responsibilities, authority, goals, and 
expectations that go with the role, in order to assess the individual’s effectiveness in performing 
those identified role-related functions. 

Understanding how deans are evaluated by faculty is a relatively new and unstudied 
phenomena (Matczynski, Lasley & Haberman, 1989). As noted earlier, researchers have been 
interested primarily in identifying the critical functions and roles of deans (Denenark, 1982; 
Morris, 1981; Morsink, 1987) and in delineating the qualities demonstrated by successful deans 
(Sivage, Bryson, & Okum, 1982). Learning to work with significant others (i.e., faculty and 
university administrators), is one of the essential tasks of deans (Matczynski, Lasley & 
Haberman, 1989). In their study of faculty’s perception of the qualities that deans should possess, 
they found that education faculty ranked communication skills as the most important skill, and 
affirmative action as the least important skill. Faculty seek a dean who exhibits a capacity to 
articulate the unit’s mission and to define the purposes of the unit and the major issues of the 
profession to various constituencies. Faculty also feel that deans should be held accountable for 
the academic standards of the unit. They expect deans to recruit high quality faculty and to 
ensure that faculty maintain high academic standards in their classrooms. Senior administrators 
seem to have similar criteria for evaluating deans (Lasley & Haberman, 1987). When Vice 
Presidents or Vice Chancellors of Academic Affairs were asked to evaluate the performance 
criteria for successful deans, they responded by saying that the success or failure of a dean [of 
education] is contingent upon several factors: faculty expertise, economic resources, and social 
circumstances. 
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The Dilemma of Deaning . The deanship offers great challenges and opportunities. As an 
administrator in the middle, the dean must learn how to work with a range of interests, 
individuals, and groups. They essentially serve two masters: the faculty and the senior 
administration. Morris (1981) describes the faculty as fiercely idiosyncratic and independent in 
their daily behavior. He believes that the nature of their work engenders a special pride in not 
being responsive to institutional rules and regulations. Thus, faculty represent a constituency that 
is almost purely political in character; that is, they cannot be commanded or led, except by the 
initiatives and cohesion of their own membership, not by a dean (p. 1 19). Morris suggests that 
because of the nature of their work, faculty are not expected to be familiar with the 
administrative perspective of the university. He contends that the perception of a chain-of- 
command mentality is an object of faculty scorn. 

Senior administration typically includes the president, provost, academic vice president, 
and the vice chancellor. The academic vice-president (or provost) relies on the dean to carry out 
the academic mission of the dean’s academic unit. It is clear that both the academic vice- 
president and the dean are pushed and pulled by many conflicting forces and demands. Tucker 
and Bryan (1991) contend that in all the various ways in which the dean and the academic vice- 
president interact, the dean is always on trial. They argue that deans are constantly judged by 
their actions and reactions to the problems, opportunities, and challenges they face (p. 197). This 
impromptu evaluation of the quality of the dean’s leadership is confirmed and reconfirmed over a 
period of several years of such interactions with senior administration (i.e., one-on-one, council 
meetings). In addition to senior-level assessments of the dean’s performance, subordinate 
stakeholders within the dean’s unit are also evaluating the effectiveness of the dean’s leadership. 
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Therefore, in an effort to move beyond such anecdotal evidence, the intent of this study is to 
empirically measure those dimensions of leadership that relate to deans’ effectiveness as 
perceived by the faculty and administrative staff who interact with them. 

The Proposed Conceptual Model 

The “transactional” model of leadership, developed largely from a social exchange 
perspective, emphasizes the implicit relational qualities of the transaction that exist between 
leaders and followers, which, in turn, yields perceptions of effectiveness (Hollander, 1964, 1978; 
Hollander & Julian, 1969, 1970). Hollander’s (1978) transactional approach to leadership 
involves the relationship of three elements, each complex within itself. These elements include 
the leader, the followers, and the situational or behavioral processes that exist between them. 

This transactional approach to leadership involves the leader 's own competencies, motivations, 
characteristics, legitimacy, and the ability to define organizational situations; the followers 
expectations of the leader’s competencies, motivations, and characteristics; and behavior that the 
leader exhibits conducting tasks, allocating resources, enforcing institutional and unit rules and 
policies, and supporting the culture, history, and mission of the organization. 

Hollander (1978) contends that followers will accept and tolerate a leader’s behavior that 
deviates from their expectations more readily if the leader engages in actions that demonstrate 
expertise and conformity to the group’s norms. This constitutes a social exchange, that is, a 
process of mutual influence and facilitating, rather than directing, the work of followers. Social 
exchange enables the leadership of highly educated professionals-those who are not likely to be 
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led. This exchange seems particularly relevant to understanding the influence of deans in 
academic organizations and faculty who rarely consider themselves followers. 

Extending our ability to measure effectiveness may also inform our theoretical 
understanding of leadership. Transactional social exchange allows us to think about how 
leadership is negotiated as a mutual and reciprocal process between leaders and followers-a 
process that responds to the mutual needs and wants of both leaders and followers. Figure 1 
provides a theoretical representation of the transactional social exchange model as applied to 
higher education. The model illustrates the reciprocal exchange processes that exist between the 
leaders (in this case, deans), followers (in this case, faculty and staff), faculty and staff 
perceptions of the dean’s leadership behavior, and in turn, faculty and staff judgments of the 
dean’s effectiveness. The model also suggests that the behavior and perceptions of faculty and 
staff may well affect the dean’s behavior and, in turn, the perceptions of his or her effectiveness. 
The use of transactional social exchange seems appropriate to explore the organizational, 
individual, and behavioral meanings that may explain deans’ effective leadership. 



Figure 1. Theoretical Model of Transactional Social Exchange 



The Proposed Multilevel Model 
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Our concern in this study is to further our understanding of how individuals who are 
nested within units, or subunits, may perceive the actions of their leaders. In the past, research 
has been hampered by a lack of analytical tools that can adequately address the various 
hierarchical structures and the complexity of interrelationships that comprise organizational 
processes. As Johnsrud, Heck and Rosser (2000) note, previous research conducted at a single 
level of analysis offered few options for modeling this complexity of organizational processes. In 
contrast, multilevel analysis provides several conceptual and technical advantages for exploring 
the leadership effectiveness of university deans. First, it provides a more refined environment to 
test theoretical relationships because the variables comprising the model can be specified at the 
correct organizational levels (i.e., individual level, group level). As previous research has 
suggested (Yukl 1989; Meindl, 1990; Hollander, 1993) leadership perceptions have an individual 
and group component. Multilevel modeling provides a framework to specify which variables 
belong to which level, and which direct effects and cross-level interaction effects can be expected 
(Hox, 1995). In the case of evaluating personnel, it becomes important to differentiate those 
variables that affect individual perceptions of leadership as opposed to those that reflect the 
group’s collective view. 

A second advantage is that multilevel modeling provides an estimate of the extent to 
which individuals in a particular context all share a similar view of that context and a means to 
incorporate that similarity directly into the analysis. Because individuals are clustered in groups, 
they share some common characteristics (e.g., values, perceptions, experiences). Ignoring the 
presence of these clustering effects (or similarities among individuals such as belonging to a 
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college or a department) in the structure of the data can lead to a biased interpretation of the 
leader’s performance. 

Third, the multilevel model provides greater precision in accounting for measurement 
error within and between units (e.g., estimates of standard errors, measurement errors associated 
with observed variables). Individuals may react cognitively, affectively, and behaviorally in 
accordance with their own definitions of organizational leadership. From this standpoint, 
leadership effectiveness is likely perceived somewhat differently by each individual faculty or 
staff member. Earlier analyses that aggregated individual responses to the group level failed to 
acknowledge the within-group variability and therefore distorted relationships examined between 
groups. In multilevel modeling, however, within-group performance assessments may be 
adjusted for individual faculty and staff members’ differing perceptions in evaluating each dean. 
After these adjustments, the between-group assessments represent the “true-score” variability 
(i.e., variance due to differences in effectiveness among persons) after the differences in 
individuals’ perceptions and measurement errors have been removed (Cronbach et ah, 1972; 
Muthen, 1994; Shavelson & Webb, 1991). The goal of the multilevel analysis is to separate the 
true differences in leadership effectiveness between deans from individuals’ varied perceptions 
of their own dean’s effectiveness and other potential sources of measurement error associated 
with individuals and groups. 

Currently, little research exists that can simultaneously assess the effective leadership of 
administrators from the perceptions of the group and the individuals within the groups (Heck, 
Johnsrud, & Rosser, in press). As shown in Figure 2, the proposed model of leadership 
effectiveness posits that structural and demographic characteristics of individual faculty and staff 
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may affect their perceptions of leadership to some extent. Structural variables could include 
one’s role (e.g., faculty, staff, department chair) and status (e.g., academic rank, tenure status, 
administrative position) within the organization. For example, we might expect department 
chairs to rate deans in more positive terms due to their closeness of proximity to the dean. 
Similarly, individual demographic characteristics such as sex and race/ethnicity may influence 
perceptions. For example, there may be a tendency on the part of faculty and staff to rate leaders 
as more effective if they are the same sex and/or ethnicity as the person doing the rating. 
Alternatively, it may be possible that different groups of faculty have differential experiences 
with the university’s administrative structure (Johnsrud .& Heck, 1994). At the individual-level 
the variables are included as controls for the perceptual differences among the individual faculty 
and staff based on their demographics. 




Figure 2. The Proposed Multilevel Model 
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On the other hand, at the group level, structural variables may explain differences in each 
dean’s effectiveness relative to the set of deans. For example, the amount of resources (e.g., 
external, instructional, non-instructional) allocated to the unit may affect differing perceptions of 
leadership effectiveness. At this level, we are providing a test of the notion that deans’ 
effectiveness is in part related to their ability to gamer valued resources for their unit (Chemers, 
1993; Fiedler & Garcia, 1987). There is pressure on deans to secure external resources to 
supplement decreased general fund allocations. Prestige and power may also accrue to units that 
can increase faculty FTEs, as this allows increased student enrollments, more faculty research 
productivity, and increased course selection. We might therefore expect deans in units with 
greater external dollars and larger FTEs to be perceived as more effective. The type of unit (e.g., 
organized research group, professional schools, arts and sciences) is also included as a structural 
variable to determine whether the context in which deans work is related to the assessment of 
their effectiveness. Finally, the demographics of the dean (e.g., sex, years of experience) may 
also have an impact on their perceived effectiveness. 



Method 



Sample 



Faculty and staff were asked to evaluate the performance of their deans based on several 
dimensions of their leadership role. In an effort to determine each dean’s overall effectiveness, 
surveys were mailed to all 1,950 faculty and staff members at a major public research university 
reporting to the 22 deans. The units consisted of various colleges, schools, and programs within 
the areas of arts and sciences, professional schools, organized research groups, and service and 
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support areas headed by deans or in some cases, directors (In this study, only those directors 
were included who hold positions equivalent to deans and who report directly to vice-presidents). 
Three mailings yielded 865 usable responses for a 54 percent return rate. 

The respondents in this study consisted of full-time faculty and administrative staff 
members reporting to selected deans. The faculty were classified as instructors, researchers, 
specialists, and librarians. The administrative staff consisted of the deans’ executive/managerial 
staff, administrative, professional and technical staff, and clerical/secretarial employees. The 
academic rank/staff position categories held by the respondents consisted of 68 (7.9%) 
instructors, 116 (13.4%) assistant professors, 128 (14.8%) associate professors, 243 (28.1%) full 
professors, 15 (1.7%) managerial/executives, 150 (17.3%) administrative, professional, and 
technical staff, and 89 (10.3%) clerical/secretarial or civil service employees. 

Sex of the respondents was fairly balanced with 405 (46.8%) females and 451 (52.1%) 
males. The race and or ethnicity reported by the respondents indicated there were 383 (44.3%) 
faculty and staff classified as minorities (African-American, Chinese, Filipino, Japanese, Korean, 
Native American, Pacific Islander, and mixed/other) and 456 (52.7%) are non-minority 
(Caucasian). The faculty and administrative staff respondents in this study proportionately 
represent the demographic populations of the institution (University Faculty and Staff Report, 
Fall 1999). 

Instrumentation and Variables 

The instrument was designed to gather information about deans’ effectiveness in 
fulfilling their leadership roles and responsibilities. The following seven domains of 
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responsibility were developed from the professional literature, on deans and a review of existing 
evaluation instruments: Vision and goal setting, management of the unit, interpersonal 
relationships, communication skills, research/professional/community endeavors, quality of the 
unit s education, and support for institutional diversity. The leadership domains were defined by 
58 Likert-type items (5-point scales). A response of “1" indicated the respondent had an 
unsatisfactory perception of the dean’s performance on that item, and a response of “5" indicated 
an outstanding level of performance on that item. A choice of either NA (not applicable) or DK 
(don’t know) was also available as an answer on each item. 

Analyses were conducted on the data in order to determine the instrument’s reliability. 
One common approach to the estimation of reliability is to examine a domain’s internal 
consistency. The alpha coefficients (in parentheses) calculated for each scale were as follows: 
vision and goal setting (.98), management of the unit (.97), support for institutional diversity 
(.97), interpersonal relationships (.98), communication skills (.98), research/professional 
endeavors (.97), and the quality of education in the unit (.98). All items from the instrument that 
comprise each scale (construct) were retained and considered reliable for further analysis. 

In addition to the scaled leadership items, individual-level (within-group) demographic 
characteristics of the respondents were also collected. The variables included sex, minority 
status, and years worked with the dean. The individual-level structural variables included 
organizational role (department chair, faculty, or staff), faculty rank (instructors, assistants, 
associates, and full professors) and staff member position. Dummy coding (e.g., females=l and 
males=0) was used as needed to handle the categorical nature of the demographic data. 




18 



17 



Several variables at the organizational-level (between-group) were also included in the 
analysis. Structural variables included: the amount of resources allocated to the unit (i.e., external 
dollars, instructional dollars, non-instructional dollars), size of the unit (i.e., FTEs of faculty and 
staff), and the type of group unit (e.g., professional schools, organized research, arts and 
sciences). There were two demographic variables at the organizational-level: sex of the dean 
leading the unit, and years in the position as dean. As conceptualized in Figure 3, the proposed 
operational model is designed to simultaneously investigate the effect of structural and 
demographic characteristics on the domains of effective leadership as perceived by the 
respondents at the individual-level (within) and organizational-level (between). 



Figure 3. The Proposed Operational Model 



Intraclass Correlations 
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The first step is to determine whether a multilevel analysis is warranted. To do so, we 
calculate intraclass correlations (ICCs), which describe the percentage of total variance in each 
leadership domain that lies between groups. 1 More specifically, larger coefficients suggest 
greater “true-score” differences (i.e., differences in effectiveness not attributable to errors 
associated with individual raters) in leadership between individual deans (Heck et al., in press). 
The results in Table 1 indicate that a substantial amount of variance in the management of the 
unit, interpersonal skills, and research/professional endeavors exists between deans (ICC=.157, 
.128, and .159, respectively). This suggests that this set of deans differs more in terms of these 
domains of leadership. Smaller true-score variance is associated with vision and goal setting, 
communication skills, quality of education, and support for institutional diversity (ICC=.099, 
.091, .068, .091, respectively). As a group, these deans are more similar in these leadership 
domains. 



Table 1. Intraclass Correlations (ICC) by Leadership Domain 



Leadership Domain 




SD 


ICC 


Vision and goal setting 


3.87 


.95 


.099 


Management of the Unit 


3.83 


.92 


.157 


Interpersonal Relationships 


3.91 


.90 


.128 


Communications Skills 


4.05 


.79 


.092 


Research/Professional/Community/ 
Campus Endeavors 


3.89 


.70 


.159 


Quality of Education 


3.86 


.84 


.068 


Support for Institutional Diversity 


4.00 


.77 


.091 
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Given there are differences between deans, the preliminary results in Table 1 represent 
the multilevel measurement and structural models of effective leadership proposed to address 
both the individual-level and group-level properties of the data. More specifically, the ICCs 
representing each of the domains (ranging from about 7% to 16%) indicate that leadership 
variability resides both within and between the units. As suggested previously, it is also 
important to emphasize that the observed variables in Table 1 are uncorrected for error. As 
Muthen (1994) notes, defining leadership effectiveness as a latent factor allows us to correct the 
observed measures for differing errors associated with each ICC (i.e., error associated with the 
similarity in perception among members of each unit) and reliability (i.e., errors associated with 
the items measured at the individual and group levels). Thus, defining leadership effectiveness as 
a multilevel construct provides results that would be given from perfectly reliable observed 
measures (Muthen, 1994). These adjustments for error will further provide more accurate 
estimates of the model’s structural parameters. 

The Multilevel Structural Equation Model 

After determining that sufficient variance in leadership exists between groups, the next 
step is to define and measure the within- and between-group variance in the leadership 
effectiveness factor and to extend the model to include sets of predictors at both levels. Because 
underlying constructs cannot be directly measured (e.g., management, communication), they 
must be indirectly defined through a set of observed variables. Multilevel structural equation 
modeling (SEM) offers an advancement in the ability to simultaneously define multidimensional 
constructs such as leadership (i.e., through confirmatory factor analysis) and then test for the 
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effects of group, demographic, and organizational role variables on the constructs. The validity 
of this proposed multilevel model is examined with Mplus 1.04 (Muthen & Muthen, 1998) using 
the maximum likelihood fitting function. In the multilevel SEM approach to testing models, one 
tests the variance-covariance matrix implied by the model against the variance-covariance matrix 
of the actual data. 

The fit of the multilevel confirmatory factor model can be assessed by the chi-square (x 2 ) 
test of model fit and the root-mean-square error of approximation (RMSEA). The chi-square is 
an inferential index that represents a test statistic of the fit of the model and is used when testing 
the null hypothesis that the proposed model fits the analyzed covariance matrix perfectly 
(Raykov & Marcoulides, 2000, p. 36). On the other hand, the RMSEA is a fit index that is widely 
used because it allows for a discrepancy of fit per degree of freedom in the model; that is, models 
with more parameters are favored in this index. The RMSEA index should be close to zero for a 
good fitting model. 

The chi-square coefficient is 260.374 (100 df) and is significant (p=.000). Although the 
chi-square coefficient is significant, the x 2 to degrees of freedom ratio is 2.6 to 1. As a general 
rule of thumb, a ratio under 3 to 1 is considered excellent for a data set of this considerable size. 
The RMSEA value is .044, which is non-significant (p=.914). The results of this index suggest 
an excellent fit of the proposed multilevel model to the observed data. With unbalanced group 
sizes, however, these coefficients should be considered as approximate only (Muthen & Muthen, 
1998). 

As shown in Figure 4, the parameter estimates indicate that all seven domains contribute 
significantly to the measurement of a dean’s leadership effectiveness both at the within- 
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(individual) and between- (organizational) group levels. All parameter estimates of the domains 
comprising leadership effectiveness are sizable and significant at both the within-group level (.73 
to .95) as well as at the between-group level (.93 to .99). 

From the variance components comprising the leadership effectiveness factor, we can 
also determine the percentage of leadership variance that lies between groups. This between- 
deans model corresponds to the actual difference in the performance attributable to the individual 
deans after accounting for other sources of variability (Heck et al., in press). In this case, the 
actual difference in effectiveness across deans is 13 percent (not tabled). 2 This coefficient 
represents the adjustment for unreliability associated with the differing intraclass correlations on 
the observed items comprising leadership (ranging from .07 to .17). Because within-group 
measurement errors are usually larger, this causes underestimation of the true between-group 
variation (Muthen, 1994). For purposes of analyzing deans’ effectiveness, the other sources of 
variation can be considered as error variance (i.e., differences in individuals’ perceptions within 
groups, measurement errors associated with items within and between groups, possible 
interactions). 

Errors of measurement are represented in Figure 4 by short arrows. The errors associated 
with individuals’ perceptions on each item comprising leadership effectiveness range from .10 to 
.47 (not tabled). In contrast, the errors associated with group perceptions of each leadership 
domain are smaller, ranging from .01 to .14 (not tabled). These coefficients provide information 
about the quality of measurement of each item comprising effectiveness at the individual and 
group levels. The methodological advantage of multilevel modeling is that the sources of error 
are removed from the analysis before considering the differences in leadership that exist among 
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the set of deans in the study. This improves the reliability and validity of inferences made about 
the assessments (Heck et al., in press). 

Further evidence of the construct validity of the leadership composite can be provided by 
determining what individual-level and group-level variables account for faculty and staff 
perceptions of effective leadership. Figure 4 displays the magnitudes of each parameter estimate 
of the predictor variables on effective leadership. Within groups, only individuals’ organizational 
role (i.e., department chairs) has a small, but significant impact on faculty and staff perceptions 
of effective leadership (.14). This suggests that chairs view the deans’ leadership in more positive 
terms. We would not expect the other demographic variables to affect the perceptions of 
leadership effectiveness (i.e., years with dean -.02, females .03, minorities .01). 
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Figure 4. Multilevel Structural Model of Effective Leadership 
(Mplus Standardized Estimates). Note: *Estimate is significant at p<.05 

At the group-level, resources (i.e., external dollars) accrued to the unit (.39), and size of 
the deans’ unit (FTEs) are significant predictors of the perceptions of effective leadership. We 
would expect these to be related to differences in leadership effectiveness. There is, however, no 
significant effect of instructional and non-instructional dollars allocated to the unit or type of 
group unit (e.g., organized research groups, professional schools) on the perceived effectiveness 
of leadership. Although we would expect these latter types of resources to be related to the 
perception of effectiveness, it may be that deans are not perceived to have the same amount of 
control over these funds. Sex of the dean (i.e., female deans) also has a significant and positive 
impact on faculty and staff perceptions of leadership (.41). 

While being cautious about the group sample size (22 dean units), the overall adequacy of 
the model clearly supports its construct validity in accounting for both group and individual 
variability in effective leadership. As noted earlier, the results indicated that 13 percent of the 
total variance in effective leadership represents the true difference in effectiveness among these 
deans, and 87 percent of the total variance is related to individuals assessing their own dean’s 
effectiveness. The finding suggests that while the perceptions of leadership are primarily 
individually held (owing to the greater range of individual views about each particular dean), 
there is also a “collective” element (or group similarity) to these views that should be the primary 
focus of organizational studies of leadership assessment. Multilevel modeling provides a more 
refined method of examining these individual and collective perceptions about leadership. 
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Additional evidence of the model’s construct validity can be seen in the amount of 
variance in leadership perceptions accounted for at each level in the model. Errors in the 
equations (i.e., variance unaccounted for) are represented in Figure 4 by the short arrows next to 
the within-groups and between-groups effectiveness construct. As we argued previously, in 
evaluating performance or effectiveness, the concern for evaluators is with the actual differences 
between the deans that result from the analytic process. Between deans, the variance accounted 
for is 57 percent (unaccounted is 43%). This provides preliminary evidence that the model (and, 
hence, the instrument) discriminates more and less effective deans on the basis of valued group 
outcomes such as FTE size and external dollars. The model also discriminates more and less 
effective deans on the basis of their sex. 

For the within-groups model (i.e., individuals assessing their own dean), only two percent 
(unaccounted is 98%) of the variance in leadership is explained by the demographic information 
associated with faculty and staff. Actually, in assessing effectiveness within groups, our primary 
concern is not with accounting for variance, as the variability in effectiveness due to individuals’ 
perceptions of their own dean is treated as error in the model. This is because each group of 
individuals only assesses their own dean (as opposed to assessing all deans). In a separate 
analysis, we determined this component represents the largest source of error (over 70% of the 
unexplained variance) in this portion of the model. 

The fact that only a small amount of variance is accounted for by the predictors within 
groups can actually be viewed as “good news.” Certain groups are not experiencing deans’ 
behavior differently in any systematic way; that is, we can assume that women or other 
underrepresented minorities are not perceiving deans’ effectiveness differently based on their 
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differential experiences. On the other hand, the significance of the dean’s sex at the between- 
group level has a very different meaning in terms of the assessment of effectiveness, because this 
represents the difference in their leadership effectiveness attributable to gender, after the various 
sources of error have been controlled. 



Discussion 

There is an abundance of research and literature that attempts to define or describe what it 
takes to be an effective leader. Fundamentally, however, leadership in organizations is not about 
the attributes the leader has, but about what the leader is perceived to say and do in the social 
context of his or her unit. An important part of that social context consists of those assessments 
that individuals and groups form regarding the leader’s effectiveness through interpersonal 
transactions. Although leadership in and of itself is often perceived as ambiguous, complex, and 
multidimensional, the concept of transactional social exchange creates order by identifying the 
mutual and reciprocal processes of organizing, planning, budgeting, and resolving issues. The 
transactional relationship between leaders and followers is maintained by the quality and 
effectiveness of those social exchanges and mutual influences— often referred to as the trading of 
benefits. Much of the literature on leadership in organizations recognizes the powerful interaction 
between leaders and their followers (Bass, 1981; Fiedler & Garcia, 1987; Hallinger & Heck, 

1999; Hollander, 1978; Hollander & Julian, 1969). 

Previous research has suggested that the perceptions of these effective or ineffective 
leadership interactions are socially constructed as an individual or group property of the social 
organization. Until recently, organizational researchers have been limited in their ability to 
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analyze their data either at the individual or group (organizational) level. Using a multilevel data 
analytical approach provides clarity and understanding in the definition and measurement of 
those interpersonal transactions that exist between leaders and followers within complex 
organizations. The findings from this research clearly support the view that effective leadership 
can be empirically defined and measured at both the individual and organizational levels. Not 
surprisingly, most of the variation in leadership effectiveness is related to individuals’ somewhat 
unique views of and experiences with their dean. More importantly, after accounting for those 
individual differences in the assessment process, it is possible to ascertain the real differences in 
effectiveness that exist between deans. 

In the case of academic deans, it is critical for them to understand what faculty and staff 
collectively perceive as important criteria to be an effective leader. Individual and group 
exchanges with the dean are constantly taking place within the organizational unit. The dean’s 
behaviors and interactions clearly affect perceptions of leadership effectiveness at the group 
level. More specifically, larger unit size (number of FTEs) and greater resources generated 
(external dollars) translate into stronger group perceptions of leadership effectiveness. In the 
tough fiscal times experienced by this university, it may be that the replacement of positions is 
seen as critical to the college’s productivity and performance (e.g., graduation rates, student 
advising and career placement, faculty publications). It is likely that the dean of the unit is 
viewed favorably for maintaining or replacing current personnel levels, or at least minimizing the 
increased pressure to perform with limited resources. On the other hand, larger units may also 
have greater flexibility to absorb reductions in staff and expenses more readily than smaller units. 
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Similarly, the resources generated by the unit in the form of external dollars is positively 
related to the group perception of effectiveness. Interestingly, dollars allocated from the central 
budget do not seem to influence the perception of the dean. Whether the external dollars are 
generated by the faculty or the dean, the dean has the authority to disseminate the overhead from 
these external dollars throughout the unit. Members within the unit can interpret the acquisition 
of valued resources as a reward and benefit for their contribution to the goals and mission of the 
unit, or members within the same unit may perceive the allocation of resources as a threat to their 
professional worklives. In the first case, the dean’s leadership may be viewed as supportive and 
effective; in the latter case, the dean may be perceived as ineffective in the equitable 
dissemination of unit resources. In this case, we provide support demonstrating that perceptions 
of leadership effectiveness are related to the attainment of valued outcomes such as resources. 
This provides evidence of the construct validity of our definition of effective leadership. 

Perhaps the most intriguing finding in this study is the relationship between the sex of the 
dean and leadership effectiveness. After controlling for several sources of within- and between- 
group variation (e.g., sex and race/ethnicity of the respondents, FTEs and external dollars, 
measurement errors), the results indicated that female deans were rated as more effective leaders. 
This finding parallels previous research conducted on the effectiveness of school principals— 
suggesting that female principals are rated as more effective leaders than their male counterparts 
(Heck, 1 995). These results contradict some research in other fields about women rating 
supervisors more favorably than men, or women tending to rate women tougher (Billing & 
Alvesson, 1994; Eagly et al., 1992; Highman, 1985). In this case, the specific multilevel analysis 
helps untangle a debate in the organizational literature. Further research, however, is suggested in 
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this area to determine why women are rated as more effective leaders— researchers should strive 
to define and measure those domains that make a difference in the way women, as well as men, 
lead their academic units. 

At the individual-level, the dean’s perceived effectiveness is influenced by the 
individual’s placement within the structure of the organizational unit, indicating that a faculty 
member’s role (e.g., department chair) in the unit is affected by the exchange process. To 
illustrate, chairs rate the deans as more effective than faculty rate them. Department chairs may 
have a greater understanding of or appreciation for what the dean does, or they may receive 
greater rewards from them in the organization. This finding may be an example of what 
Hollander (1978) refers to as the dissemination of privileged information and communication. 
The chair has a closer interpersonal relationship and line of communication with the dean than 
most faculty and administrative staff within the unit or department. Moreover, Cook (1982) also 
refers to this as the intensity and proximity of the actors within the process of a social exchange. 
In this study, other individual demographics of the respondents such as sex and minority status 
had little or no impact on the perceptions of these exchange relationships. We would not want 
these to affect the perceptions of leadership effectiveness, and in this case, they do not. 

The use and application of transactional social exchange in this research has provided a 
conceptual framework to consider the exchanges that happen between deans and their faculty and 
staff in terms of power and authority, and benefits and rewards. The significant findings in this 
study are related to these desired ends (resources and size which bring power and benefits). The 
perspective that these collective transactions influence both the receiver and the sender in the 
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exchange process provides a better understanding of the importance of the reciprocal 
relationships that exist between deans and their faculty and staff. 

Leadership, with its multiple theoretical approaches and applications, remains a powerful 
phenomenon, and our understanding of leadership within the complexities of social organizations 
continues to evolve. Leadership effectiveness in this study has been defined and measured as one 
construct comprised of seven domains that represent the roles, responsibilities, and functions of 
the dean’s leadership. Prior research on leadership evaluation in academic organizations has 
focused primarily on single-level analyses that involve the leader’s perception of their own 
performance or effectiveness or the perception of their performance as evaluated by their 
superiors. New techniques, such as the multilevel SEM approach used in this study, can provide 
researchers with an important means to define and measure social processes and behaviors that 
exist within multiple levels of the organization; that is, this approach differentiates individuals’ 
perceptions from the group’s collective view. Moreover, at the group-level we demonstrate that 
the level of effectiveness is related to valued resources, providing support for the trading of 
benefits that takes place between deans and their faculty and staff. Multilevel modeling with 
latent variables also allows more refined measurement, which provides more accurate results 
(i.e., structural paths that are corrected for error). 

Conclusions 

Most of the theoretical work to date suggests that effective leadership is best conceived as 
a multidimensional construct consisting of a variety of leader-follower exchange processes. It is 
important, therefore, to study how leaders, through their social interactions and behaviors, 
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influence individuals within the social systems they oversee. Throughout this study, a primary 
concern has been to further our understanding of how individuals and groups perceive the 
behaviors and processes of their leaders. Institutions can only benefit in mapping the domains of 
the critical social exchanges that exist between leaders and their followers. This should lead to 
more effective evaluation of leadership performance. 

What this study suggests is that leaders, or deans, through their social interactions, need 
to be aware of the effectiveness of their transactions of social exchange— an effectiveness that 
comprises the ability to gamer individual and group support, to motivate people beyond their 
own expectations, to conceptualize and pull ideas together, to exemplify good taste and 
judgment, and to possess the leadership savvy to perform the various functions, tasks, and duties 
in a manner and style reflective of the organization’s goals and mission. In pursuit of 
effectiveness, the academic leader needs to bring to bear an understanding of the complexities of 
social exchange within an academic setting and attend to the perceptions that individuals and 
groups form regarding their performance. 
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Endnotes 

1 .The intraclass correlation is described as the proportion of the variance in an outcome that lies 
between groups (deans): 

= oo /( oo + 2 )• 



2. We can estimate the proportion of the leadership variance that lies between-deans (\|/ B ) relative 
to the total leadership variance (y T ). This corresponds to an adjustment made for the individual 
measurement properties of the observed variables comprising the leadership factor (e.g., differing 
intraclass correlations). The latent variable counterpart of an intraclass correlation for observed 
variables can therefore be expressed as 
Yb / (V b + Vw)« 
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